SPEARMAN’S  FOOTRULE  AS  A MEASURE  OF  DISARRAY 
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1. 


Introduction. 


Widely  used  nonparametric  measures  of  association  such  as  Kendall's  x 

and  Spearman's  p lead  to  natural  metrics  on  the  set  Sr  of  permutations 

of  n letters.  Let  ft  and  cr  be  elements  of  S . Four  metrics  which 

n 

we  consider  are 


(1-1) 


(1-2) 


n 

D(rtjO-)  = I |rt(i)-ff(i)| 
i=l 


S (rt ,cr ) = E (ft(i  W(i  ) )2 
i=l 


(1-3 ) 


T(n,a)  = the  minimum  number  of  transpositions  required 
to  bring  (ft (1), . . . ,ft(n) ) into  the  order  (cr(l),. . . ,cr(n). 


(1-4)  l(n,cr)  = the  minimum  number  of  pairwise  adjacent  transpositions 

required  to  bring  (ft_1(l ), . . . ,ft-1 (n > ) into  the  order 
(ct_1(1  ), . . . ,<r  1(n)).  Here  ft"1  and  cr  1 are  the  permutations 
inverse  to  ft  and  cr. 


In  section  2 we  derive  the  mean,  variance  and  limiting  normality  of 
D(ft,cr)  when  ft  and  cr  are  chosen  independently  and  uniformly  frcm  SQ. 
Similar,  known,  results  for  S,  T and  I are  summarized  in  Table  1.  In 
section  3 we  prove  that  I+T  < D < 21.  The  proof  and  subsequent  discussion 
of  boundary  cases  depends  on  combinatorial  arguments  of  independent  interest. 
In  section  4 we  discuss  the  statistical  implications  of  section  2 and  3 along 
with  some  open  problems. 


2.  Properties  of  the  4 Metrics. 

If  p denotes  one  of  the  functions  D,  S,  T or  I,  then  it  is  easy 
to  verify  that  p is  a metric  on  Sn;  that  is: 

p(rc,cr)  > 0 and  p(n,cr)  =0  if  and  only  if  n = cr  , 

p(n,u)  = p(cr,n)  and  p(n,<r)  < p(«,  tj)  + p(rj,it)  . 

Further,  all  four  metrics  are  right  invariant  in  the  sense  that 
p(n,cr)  = p(^,oti)  for  tj  e Sn-  In  particular  p(l,n)  = p(l,«_1), 
where  we  write  1 for  the  identity  permutation  in  S^.  We  shall  usually 
abbreviate  p(l,n)  by  p(n).  We  consider  more  specific  properties  below. 


D(fl,cr):  The  sum  of  the  absolute  values  of  the  difference  in  the  ranks  is 

the  metric  associated  with  Spearman's  footrule.  Kendall  (1970)  discusses 
the  footrule  as  a measure  of  association  and  dismisses  it  because  of  a lack 
of  available  statistical  properties.  Toward  this  end  we  prove 

Theorem  1.  Let  * and  a be  permutations  chosen  independently  and 
uniformly  in  SR.  Hien,  as  n goes  to  infinity 


(2-1 ; 


E{D(n,a))  = + 0(n) 


(2-2] 


(2-3) 


Var(D(fl,cr)}  = r?  + 0(n2) 

1 1 J 1 


■»o 


where  $ (x ) 


f 

J- 


X 2 /_ 

/2<u  . 
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Proof.  Using  right  invariance,  the  distribution  of  D(n,a)  is  the  same 

as  the  distribution  of  D(l,n)  = D(n).  Hius  we  may  restrict  our  attention 
to  D(jt). 

We  first  compute 

n n 2 

E(D(«))  - X - X |i-jl  = -r  + 0(n)  . 
i=l  n j=l 

This  proves  (2-1).  Hie  proof  of  (2-2)  is  a lengthy  computation  and  we 
only  sketch  the  details.  Consider 

n 

(2-U)  var{D(«))  = X var|i-*(i)|  + 2S  cov( | i-n  (i ) I , | (j  ) I ) 

i=l  i <3 


= + o(n2)  + 2 X cov(|i-*(i)| , i d-n (J ) I ) • 

^ i<  j 

To  simplify  the  evaluation  of  the  covariance  sum  note  that  for  any  bounded 
functions  f and  g, 

E(f(*(i))g(*(d))}  = f(£)e(k) 

= E{f(«(l)))E(g(n(l)))  - E(f(n(l))g(it(l))J 

= Ef(ir(l))Eg(«(l))  - cov(f  (« (l ) ),g  (« (1 ) ) ) • 

In  particular, 

(2-5)  cov(|i-it(i)|,|j-n(j)|  ) = - cov(|i-*(l)| , I ) • 
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Consider 


(2-6) 


cov(|i-*(l)U  J-n (1 ) | ) = — Z | i-£  | | 

n i=  1 


- — (i(i-l)+n(n-i-l)}  { j(j-l)+(n-j  ) (n-j-1)} 


4n 


= f^^j)  + f2(i,j)  . 


Summing  f.(i,j)  in  i < j for  fixed  j and  then  in  j yields 


(2-7) 


Z f (i,j)  = + 0(n3) 

i < j ^ 


Similarly 


(2-8) 


i<J 


f2(i,j)  = - JB  + °(“5) 


(2-8)  and  (2-7)  along  with  (2-6)  and  (2-5)  show  that 


I cov(|i-n(i)|J  j-n(j)|  ) = - r^r  + 0(n2 
i<  J 3 


Using  this  in  (2-4)  completes  the  proof  of  (2-2). 

To  prove  (2-5)  we  use  HoefCdings  (1951)  combinatorial  central  limit 

theorem.  This  states  that  if  (a"j)  i,j=l,...,n  is  a sequence  of  square 

n n 

array's  and  if  W = T,  a.  , where  * is  a random  permutation  in 
n i=i  i*(i) 

Sn  then,  subject  to  growth  conditions  on  a^j  , 


lim 

n-$> 


Using  (2-2)  we  readily  verify  the  sufficient  condition  given  in  equation 
(12 ) of  Hoeffding  (1951 ) for  the  array 

= | i — J | i> j~l* • • • >n  . 

This  completes  the  proof  of  theorem  1.  [] 

S(ir>g):  The  sum  of  squares  of  the  difference  in  the  ranks  is  the  metric 

associated  with  Spearman's  coefficient  of  association  p.  All  of  the 
results  listed  in  table  1 may  be  found  in  Kendall  (1970). 

T(ft,g):  This  seemingly  natural  metric  has  not  received  much  attention  in 

the  statistical  literature.  Note  first  that  if  \|r  and  q are  in  Sn, 
we  have 

T(jt,ct)  = T ('jnwj } ijrtyrj ) = t(1»*  V)  = T(*  10’)  . 

A result  due  to  Caley  (1849)  states  that  T(n_1cr)  = n-C(Tt_1g)  where 
C (q ) is  the  number  of  cycles  in  q.  This  gives  an  easy  way  to  compute 
T(n,cr).  Feller  (1968),  pg.  256,  gives  a representation  of  the  number  of 
cycles  in  a random  permutation  as  a sum  of  independent  random  variables. 

This  representation  and  standard  theorems  from  probability  theory  easily 
imply  the  limiting  normality  of  T as  well  as  the  results  listed  in 
table  1. 

I(fljcr) : This  metric  arises  from  Kendall's  measure  of  association  t.  It 

is  thoroughly  discussed  in  Kendall  (1970).  A simple  approach  to  the  analysis 
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of  the  stochastic  properties  of  this  metric  is  as  follows:  A permutation 

T is  said  to  have  an  inversion  at  (k ,t ) if  k < l and  11  (k)  > tj  U ). 

A standard  result  (see  for  example  Knuth  (1973)  Sec.  5- 1*1)  is  that 
l(ff,c r)  = 1(1, nc”^)  = the  number  of  inversions  in  na”1.  Feller  (1968), 
pg.  256  gives  a representation  for  the  number  of  inversions  in  a random 
permutation  as  a sum  of  independent  random  variables.  From  this  the 
liMting  normality  as  well  as  the  results  listed  in  Table  1 follow  easily. 

The  following  table  summarizes  the  means  and  variances  of  these 
metrics.  Only  the  leading  term  of  the  mean  and  variance  is  indicated, 
the  notation  [x]  is  used  for  the  integral  part  of  x. 


D 


S 


T 


I 


Table  1 


Max 

2 

l2  J 


Mean 

n2 

3 


Variance 

2 r? 

*5  n 


3 r, 

n -n 


n3 

r 


n5 

3S 


n-1 


n-log  n 


log  n 


2 

n -n 


2 

n 

r 


n3 * 5 

IS 


3.  Some  Inequalities  Involving  D(n,q). 

There  are  important  relations  between  the  various  metrics  v$lch 

generally  take  the  form  of  Inequalities  which  must  be  satisfied.  For 
example,  Kendall  (1970)  discusses  the  Durbln-Stuart  inequality,  which 
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in  our  notation  is: 


S(*,ct)  > | I(n,ff){l  + . 

We  now  prove: 

Theorem  2. 

(3-D  l(«,c r)  + T(h,it)  < D (it , cr ; < 2I(it,cr)  . 

Proof.  Since  all  the  quantities  in  (3-1)  are  right  invariant  it  is 
sufficient  to  show  that 

(3-2;  I(w)  + T(ji)  < D(ji)  < 2l(it ) . 

We  prove  the  right  hand  inequality  first. 

I(n)  = l(n  1)  is  the  smallest  number  of  pairwise  adjacent  transpositions 

required  to  bring  it  to  the  identity.  Let  x^,l<i<l(n),  be  a 
sequence  of  integers  which  index's  a sequence  of  transpositions  which  trans- 
form 1 to  it^  to  to  ...  to  it.  The  i^1  transposition  transforms 

ni  ni+i  by  inter  changing  if^(x^)  and  n^(x^+l).  We  may  assume  that 

the  sequence  x^  is  chosen  so  that  it^(x^)  < n^(x^+l).  Consider  the 
difference 

Ai+1  " D^ni+1^  “ D^i^ 

= |xjL-iti(x1+l)|+|x1+l-iti(x1)|-|x1-iti(xi)|-|x1+l-iti(xi+l)|  . 

There  axe  three  possibilities 

Case  1.  t^O^+l)  < x±.  Then  «i(xi)  < xA  and  Ai+1  = 0 . 
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Case  2.  n^(x^)  > x^+1.  Then  = 0. 

Case  3.  «i(xi+l)  > x^  l,  ni(xi)  < x±‘  Ihen  A1+1  = 2. 

I(») 

Thus  D(«)  = Z Ai<2I(jr)  as  desired. 

i-1 

To  prove  the  left  hand  side  of  the  inequality  (3-2)  we  need  some 
more  notation. 

Denote  the  inversion  n(k)  > n(i)  with  k < l by  [k$ £].  Let  us 
call  [k;i]  a Type  I inversion  if  «(k)  > i and  a Type  II  inversion  if 
n(k)  < l.  Thus,  every  inversion  of  * is  either  a Type  I or  Typb  II 
inversion  and  some  inversions  may  be  of  both  types. 

For  a fixed  k,  if  [kjy]  is  a Type  I inversion  then  we  must  have 

k < y < it(k)  . 

Therefore,  denoting  the  number  of  elements  in  a finite  set  A by  |a|. 


(3-3)  | Cy:  [k;y]  is  a Type  I inversion)!  < n(k)-k  . 

Similarly,  if  [x;£]  is  a Type  II  inversion  then 

n(i  ) < «(x)  < £ 


and  so, 

(3»^)  j(x:  [xjl  ] is  a Type  II  inversion  )|  < £-n(t)  . 

Thus,  it  follows  that 
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D(« ) = Z |k-it(k)| 

k 

= Z (n(k)-k)  + Z («-*(«)) 
if  (k)  >k  i ■(£)<£ 

(3-5) 

> # of  Type  I inversions  + § of  Type  II  inversions 

> I(n)  . 

(3-5)  is  clearly  a step  in  the  right  direction  toward  proving  (3-2). 
In  order  to  actually  prove  (3-2)  we  have  to  examine  (3-5)  more  carefully. 
In  particular  we  must  see  how  much  was  given  away  at  each  of  the  two 
inequalities  in  (3-5). 

In  the  first  place,  suppose  for  some  integer  y, 

(3-6)  k < y < n(k)  < *(y)  • 

Such  a y is  counted  by  the  sum  Z (ir(k)-k)  but  is  not  counted  in 

n(k)>k 

the  sum  of  the  numbers  of  Type  I and  Type  II  inversions  since  [k;y]  is 
not  an  inversion. 

Similarly,  if  x satisfies 

(3-7)  ir(x)  < n(g)  < x < l 

y 

then  x is  counted  by  the  sum  (l -«(£))  but  [xj£ ] is  not  an 

)<l 

inversion.  Thus,  for  each  such  x and  y we  get  a contribution  of  1 
to  D(k)  - I(«). 

The  other  place  in  (3-5)  where  we  have  a potential  gain  of  D(n) 
over  l(n)  i8  the  last  inequality.  Here,  we  gain  1 for  each  inversion 
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of  r-  which  is  both  Type  I and  Type  II,  i.e.,  for  each  x having 
(3-8)  x < k(x)  , r>£" (x)  < * (x ) . 


Let  us  split  (3 -6 ) and  (3-7)  into  two  pieces  each: 


(3-6' ) 

2 

k < n (k)  < n (k) 

(with 

y^(k)) 

(3-6") 

k < y < n(k)  < n(y) 

(3-7*  ) 

ff2  (£  ) < n(£)  < £ 

(with 

X 

il 

* 

(3-7") 

Jt(x)<x(£)<x<0  . 

Let  us  now  restrict  our  attention  to  a cycle  C = (c^o^, . . . ,c  ) 
of  ' r.  of  length  |c|  = m.  Thus,  n(ck)  = c^+1  for  1 < k < m,  and 
«(cm)  = c^.  We  call  an  element  or  pair  of  elements  of  C satisfying 
(3-6'),  (3-6"  )>  (3-7*  ),  (3-7")  or  (3-8)  a critical  configuration  of  C. 

We  now  show  that  C has  at  least  m-1  critical  configurations.  We 
proceed  to  induction  on  m.  For  m=l  there  is  nothing  to  prove.  For 
m=2  it  is  immediate  since  in  this  case  (3-8)  clearly  holds  for  the  smaller 
element  of  C.  Assume  for  some  m > 2 that  the  claim  is  valid  for  all 
values  < m and  suppose  C is  an  m-cycle  which  has  at  most  m-2  critical 
configurations.  There  are  several  cases: 

2 

Case  1.  Suppose  x e C satisfies  (3-6'),  i.e.,  x < n(x)  < « (x).  Form 
the  (m-l)-cycle  C'  by  removing  x from  C.  It  is  not  hard  to  check 
that  we  have  reduced  the  total  number  of  critical  configurations  by  at  least 
1 so  that  C*  has  at  most  m-3  critical  configurations,  which  is  a 
contradiction  to  the  induction  hypothesis. 
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Case  2.  Similarly,  if  x c C satisfies  (3-7' ) then  by  deleting  it  from  C, 


we  form  an  (m-l) -cycle  C'  having  at  most  m-3  critical  configurations, 
which  is  impossible. 

Case  3.  Suppose  there  exists  x e C satisfying 

O ~7. 

X < Tt^(xJ  < fl(x)  < ^ (x)  . 

Thus,  C has  a critical  configuration  satisfying  (3-6")  with  k-x  and 

2 2 

y=n  (x).  Form  the  (m-2; -cycle  C'  by  deleting  x(x)  and  n (x)  from 

C.  It  is  now  not  difficult  to  see  that  the  total  number  of  critical 
configurations  remaining  is  at  most  m-4  (note  that  x also  satisfies 
(3-8)),  which  contradicts  the  induction  hypothesis. 

Case  4.  A similar  argument  applies  if  some  y e C satisfies 

y > *2(y)  > *(y)  > ^ (y)  . 

Therefore,  we  may  assume  that  cases  1 through  4 do  not  occur  in 
C.  However,  if  c denotes  the  least  element  of  C then  this  assumption 
forces  n (c)  always  to  be  strictly  between  n (c)  and  * (c)  for 

any  k > 0.  This  is  clearly  impossible  (since  C is  a cycle)  and  completes 
the  induction. 

Thus,  each  m-cycle  of  n contributes  at  least  m-l  to  the  difference 
D(n)  - l(n).  Therefore 

D(«)  - I(«)  > £ (|C|-1)  = Z |c|-C(k)  = n-C(n)  . 

~ C=cycle  C 

of  n 

This  proves  Theorem  2.  □ 
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As  a numerical  example,  Table  2 lists  the  values  of  the  4 metrics 
^ien  n=4. 


Table  2. 


n 

cycles 

TOO 

100 

DOO 

sOO 

(1234) 

(l)  (2)  (3 ) (4  ) 

0 

0 

0 

0 

(1243 ) 

(1)(2)(34) 

1 

i 

2 

2 

(1324 ) 

(1)  (23  ) (4 ) 

1 

l 

2 

2 

(1342) 

(1)(234) 

2 

2 

4 

6 

(1423 ) 

(1)(243) 

2 

2 

4 

6 

(1432) 

(1)  (24)  (3 ) 

1 

3 

4 

8 

(2134) 

(12)  (3)  (4) 

1 

1 

2 

2 

(2143 ) 

(12)  (34) 

2 

2 

4 

4 

(2314) 

(123)  (4) 

2 

2 

4 

6 

(2341) 

(1234) 

3 

3 

6 

12 

(2413  ) 

(1243 ) 

3 

3 

6 

10 

(2431) 

(124)  (3) 

2 

4 

6 

14 

(3124 ) 

(132)  (4) 

2 

2 

4 

6 

(3142) 

(1342) 

3 

3 

6 

10 

(3214) 

(13)  (2)  (4) 

1 

3 

4 

8 

(3241 ) 

(134)  (2) 

2 

h 

6 

14 

(3412) 

(13) (24) 

2 

4 

8 

16 

(3421) 

(1324) 

3 

5 

8 

18 

(4123 ) 

(1432) 

3 

3 

6 

12 

(4132; 

(142)  (3) 

2 

4 

6 

14 

(4213 ) 

(143)  (2) 

2 

4 

6 

14 

(4231) 

(14)  (2)  (3) 

1 

5 

6 

18 

(4312) 

(1423) 

3 

5 

8 

18 

(4321) 

(14)  (23) 

2 

6 

8 

20 

12 


Remarks.  1.  An  iramediute  deduction  from  Theorem  , is  the  bound  C(^)  + I(rr ) > n. 


r- 


r$? 


I 


2.  We  can  easily  characterize  the  permutations  " » 3 where  Df"  takes  on 

( n 

its  maximum  value.  For  n even,  D(" ) _ with  equality  if  and  only  if 
n(i  ; > "7  for  i =1,2, . . . ,2-.  Thus  D(>T  takes  on  its  maximum  value  for 

c.  r 

) choices  of  . A similar  bound  holds  for  odd  n.  This  skewness 
partially  explains  why  the  mean  of  D(it'j  is  larger  than  ^ . 

l>.  It  is  natural  to  consider  when  equality  is  attained  in  theorem  2.  The 

example  in  table  2 shows  equality  of  the  lower  bound  in  2.5  out  of  24 

cases.  For  larger  n consideration  of  the  mean  values  shows  that 

equality  can  occur  for  at  most  o(n! ) elements  of  S . We  can 

n 

characterize  the  elements  n t Sn  for  which  equality  occurs  in  the 
upper  bound.  Let  us  say  that  n has  a 3 -inversion  if  there  exist 
* < J < k with  *(i  > > x(j)  > fl(k).  It  can  be  shown  that  D(rc)  s 2I(n) 
if  and  only  if  n has  no  3-inversions.  Khuth  (1973),  Sec.  5.1.4,  notes 
that  the  number  of  permutations  in  Sn  with  no  '-inversions  is  exactly 

the  well  known  Catalan  number  ~ — /2n+l > 

2n+l  v n ' 

4.  The  number  of  permutations  n satisfying  i(a)  = T(n)  is  easily  seen 
to  be  the  Fibonacci  number  Fgn_1  defined  by  FQ  = 0,  F = 1 and 

Fn+2  ~ Fn+1  + Fn  for  n - °*  ^is  shows  that  simultaneous  equality 
of  both  bounds  in  Theorem  2 holds  exponentially  often. 
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4.  Interpretation  of  Results  and  Some  Open  Problems. 


Statisticians  most  often  normalize  metrics  so  that  they  have  the  proper- 
ties of  a correlation  coefficient.  The  translation  is  straightforward,  if 

P is  a metric  on  S and  its  maximum  value  is  m,  define  a measure  of 

n 

association  by:  R(n,cr)  = 1 - . v/e  note  in  this  connection  the 

confusion  on  pgs.  32-3;'  in  Kendall  (1970)  concerning  Spearman's  footrule. 

2 

It  is  clear  that  the  proper  choice  of  m for  the  footrule  is  [~ ].  Knuth 
(1973 ) works  with  I and  T directly  as  measures  of  disarray. 

An  example  from  Kendall  (1970),  will  help  clarify  a discussion  of 
invariance.  Consider  a class  of  10  students  ranked  in  mathematics  and 
in  music. 


Pupil 

a 

b 

c 

d 

e 

f 

g 

h 

i 

Mathematics (x) 

7 

4 

3 

10 

6 

2 

9 

8 

1 

5 

Music  (u) 

5 

7 

3 

10 

1 

9 

6 

2 

8 

4 

A typical  use  of  the  metrics  we  suggest  would  be  to  compute  the  distance 
between  the  permutations  k and  p as  a measure  of  association.  Right 
invariance  here  refers  to  invariance  under  changes  in  the  labels  a 
through  j,  of  the  students.  Right  invariance  is  compelling  in  so  many 
situations  that  we  have  forced  all  of  the  metrics  to  be  right  invariant. 
This  accounts  for  the  at  first  unnatural  appearance  of  the  definition  of 
I(*,cr).  Left  invariance  corresponds  to  relabeling  the  ranks;  as  an 
example,  it  could  be  that  10  means  the  worst  ranking  and  1 the  best, 
instead  of  the  opposite  relation.  Among  the  metrics  we  consider,  T is 
left  invariant  as  well  as  right  invariant. 
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The  results  in  Tabic  1 suggest  that  T is  unsuitable  for  general  use, 
having  very  small  variance  about  a mean  very  close  to  its  maximum  value. 

Of  the  three  remaining  metrics  S has  the  largest  range  with  corresponding 
greater  variability.  S has  the  natural  interpretation  of  the  euclidian 
distance  between  the  vectors  « and  o regarded  as  points  on  the  surface 
of  a sphere  in  n dimensional  space.  I and  D seem  roughly  similar, 

D seems  somewhat  easier  to  interpret  directly  while  I has  the  advantage 
of  having  its  distribution  tabulated  for  small  sample  sizes  (cf.  Kendall 
[1970J).  The  inequalities  in  Section  3 also  suggest  that  the  difference 
between  I and  D is  not  very  great. 

The  quantities  I and  T arise  in  the  analysis  of  sorting  algorithms. 
In  particular  Knuth  (1973,  pg.  l4l)  considers  the  quantity  B(fl)  = the 

number  of  right  to  left  maxima  in  n.  Khuth  (1973,  pg.  157)  shows  that, 

2 

in  our  notation  2B(* ) < l(n)  + T(«)  < [tj-]. 

Use  of  theorem  2 actually  gives  the  stronger  inequality  2B(*)  < D(x). 
The  quantity  D(n)  also  appears  in  Problem  5.2. 1-7  of  Knuth  (1973)  in 
connection  with  another  sorting  algorithm. 

We  conclude  by  listing  some  open  problems. 

1.  Find  inequalities  relating  D(l,n)  to  S(l,n).  There  is  the  obvious 
bound  D(l,n)  < S(l,n).  Note  that  even  this  weak  hound  holds  exponentially 
often.  Indeed,  it  is  not  hard  to  see  that  D(n)  - S(n)  for  exactly  F 

n+1 

permutations  n € Sn  (cf.  Remark  4). 

2.  Find  reasonable  2 -sided  invariant  metrics. 

3.  Characterize  the  permutations  In  where  equality  of  the  lower  bound 
of  Theorem  2 holds. 
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